fix(retention-job): add chunking strategy for cleanup#4305
fix(retention-job): add chunking strategy for cleanup#4305TheodoreSpeaks merged 2 commits intostagingfrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
|
@BugBot review |
PR SummaryMedium Risk Overview Introduces Reviewed by Cursor Bugbot for commit 3c48b24. Bugbot is set up for automated code reviews on this repo. Configure here. |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 66a66d5. Configure here.
Greptile SummaryThis PR refactors the retention cleanup jobs to chunk large workspace-ID Confidence Score: 4/5Safe to merge — the fix addresses a real production failure with correct logic; the two P2 items are about failure-counter semantics and an uncapped per-run row budget that should be validated before deploying at scale. No P0/P1 bugs found. Two P2 issues: apps/sim/lib/cleanup/batch-delete.ts — the new Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[workspaceIds] --> B[chunkArray\n50 IDs per chunk]
B --> C{for each chunk}
C --> D[selectChunk\nSELECT up to batchSize rows]
D --> E{rows.length == 0?}
E -- yes --> F[next chunk]
E -- no --> G[onBatch?\ne.g. delete S3 files]
G --> H[DELETE rows by ID]
H --> I{rows.length == batchSize\nAND batches < maxBatches?}
I -- yes --> D
I -- no --> F
F --> C
D -. error .-> ERR[result.failed++\nskip to next chunk]
G -. error .-> ERR
H -. error .-> ERR
C -- all chunks done --> LOG[Log: total deleted\nchunk failures]
|
Summary
Add chunking cleanup strategy to avoid failures when number of cleanup items exceed maximum parameters in a sql query.
Unified logic to single helper method.
Type of Change
Testing
Checklist
Screenshots/Videos